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(57) Abstract 

A method and apparatus for controlling bit rates used in a moving pictures encoder, such as an MPEG standard encoder A sequence 
of movlg Pictured is dTvided into segments each of which comprises one or more group of pictures. A constant overall b.t rate s specified 
2rT sequence d pictures, but variable bit me encoding used within each segment. A difference between the number of bus allocated 
for IncS tne segment and me actual bits used for encoding is determined, and the difference d.stnbuted over one or more subsequent 
segments. 
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MOVING PICTURES ENCODING WITH CONSTANT OVERALL BIT-RATE 

Field of the invention 

5 The present invention relates to a method and apparatus for encoding moving pictures. In 
particular, the present invention relates to a method and apparatus for performing variable 
bit rate control in a digital video encoder while maintaining a particular overall bit-rate. 

Background of the Invention 

10 

One of the main obstacles faced by industry dealing with digital video processing, storage 
and transmission is the large amount of data needed to represent analog video in the digital 
domain. Accordingly, digital compression is often applied to moving pictures to achieve a 
reduction in required transmission bandwidth or storage size. One variety of such 
15 compression techniques can be derived from the ISO/IEC MPEG Standards, the ISO/IEC 
11172-3 (MPEG-1), the ISO/IEC 13818-2 (MPEG-2) and the MPEG-2 TM5 (test model 
5), developed by the Moving Picture Experts Group of the International Organisation for 
Standardization. The disclosures of those standards documents are hereby expressly 
incorporated into this specification by reference. MPEG-1 is the compression standard 
20 used in VideoCD while MPEG-2 is the video compression standard used in DVD and 
many digital broadcasting systems. 

The MPEG standards specify only the syntax of the compressed bitstreams and method of 
decoding. The method of implementation in the encoder is left to the developer, and any 
25 form of encoder may be employed as long as the resulting bitstream conforms with the 
specified syntax. 

In certain applications such as video storage device (recorder), it is possible to use variable 
bit rate (VBR) encoding. A VBR encoder is able to vary its output bit-rate over a larger 
30 range than a CBR encoder, and this would generate an output which has a more constant 
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visual quality. An example of a VBR encoder is described in United States patent number 
5,650.860, entitled "Adaptive Quantization". In order to maintain a maximum bit rate 
allowed by the target storage device as well as an overall bit-rate which enables input 
picture sequence to be stored into a defined storage space, such VBR encoders utilise 
5 multiple encoding passes. 

In the first encoding pass, the bit utilisation information is determined for each scene or 
each picture in the input sequence. This may be done by fixing the reference quantisation 
step size and disabling the VBV control. The determined bit utilisation information is then 

10 used to generate a bit budget for each scene or picture such that an overall target number 
of bits to code the sequence is fixed, and that the maximum bit rate is not violated. In 
cases that bit utilisation information obtained is not close to that required for generating 
the bit budget, steps from the first coding pass must be repeated with an adjusted reference 
quantisation step-size. The input sequence is coded in a final pass using the generated bit 

15 budget information to achieve the target bits or overall bit rate. 

Multiple-pass VBR encoders requires large storage memory for intermediate bit utilisation 
information, and large computation needs for the additional passes and bit budget 
generation. Furthermore, such a VBR encoder cannot process input sequences in real-time 
20 as required by certain applications. 

Summarv of the Invention 

It is an object of the present invention to provide a single-pass real-time variable bit rate 
25 encoder for moving pictures. It is also an object of the present invention to provide 
variable bit rate encoding of moving pictures such that the change in encoded picture 
quality from one scene to another is minimised. A further object is to provide a real-time 
variable bit rate encoding algorithm which produces a constant overall bit rate. 

30 In particular, the present invention encodes an input moving pictures sequence one 
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segment at a time according to a target encoding quality which is determined by a target 
segment bit rate. The target segment bit rate of a current segment is preferably derived 
from the differences between the target segment bit rates and the actual coding bit rates of 
previous or previous few encoded segments. 

5 To maintain consistent encoding quality for all pictures within a segment, the actual target 
bit rate for encoding the pictures is made variable according to their scene completes as 
well as the target encoding quality of the segment. 

10 As the target encoding quality of each segment is modified based on the differences 

between the target and the actual bit rates of previous or previous few encoded segments, 
the change of target encoding quality from segment to segment is made relatively smooth 
compared to that of a Constant Bit-Rate Encoder, and furthermore, the overall bit rate of 
encoding is maintained constant 



15 



In accordance with the present invention, there is provided a method for use in a moving 
pictures encoder for encoding a sequence of segments each having at least one image, 

comprising the steps of: 

a) determining an overall target bit rate for encoding the sequence of images; 
20 b) deterrnining a bit allocation and target quantisation step size for encoding a 

first segment on the basis of a segment target bit rate calculated using said overall 
target bit rate; 

c) encoding said first segment using a variable bit rate encoding method 
according to the target quantisation step size; 

d) determining a difference between the number of bits used to encode said 
first segment and said first segment bit allocation^ 

e) distribuungTaid difference for use in encodlng~aTleist one subsequent 
segment to determine a subsequent segment bit allocation; 

f) determining a new target quantisation step size for encoding a said 
subsequent segment on the basis of a new target segment bit rate calculated using 



25 



30 



SUBSTITUTE SHEET ( rule 26 ) 

BNSDOCID: <WO 9949664A1_I_> 



WO 99/49664 



PCT/SG98/00022 



-4- 

said segment target bit rate and the distributed difference; and 

g) encoding said subsequent segment using a variable bit rate encoding method 

according to the new target quantisation step size; 

wherein variable bit rate encoding is employed for encoding pictures within a 
5 segment whilst maintaining a substantially constant bit rate over said sequence. 

The present invention also provides a method for encoding moving pictures in a moving 
pictures encoder wherein a sequence of images are provided as input, the sequence of 
images comprising a plurality of segments each having a plurality of images, the method 
10 including: 

a) determining an overall target bit rate for encoding the sequence of images; 

b) maintaining a distribution record of bits from at least one previously 
encoded segment allocated for use in encoding at least one segment to be encoded; 

c) determining a target segment bit rate for a segment of the sequence of 
15 images on the basis of the overall target bit rate and a bit rate change calculated 

from the corresponding allocated bits from the distribution record; 

d) determining a target segment encoding quality from the target segment bit 
rate, a preceding target segment bit rate and a preceding target segment encoding 
quality,; and 

20 e) encoding the images of the segment according to the target segment 

encoding quality using a variable bit rate encoding technique taking into account 
scene complexities of the images in the segment; 

wherein maintaining said distribution record includes determining a difference 
between the number of bits used to encode a particular segment and the number of bits 
25 allocated for encoding the particular segment on the basis of the target segment encoding 
quality. 

According to the current invention, a moving pictures sequence is divided into segments. 
The size of each segment may be suitably determined. Each segment is encoded with a 
30 target encoding quality derived from its target segment bit rate. A variable bit rate (VBR) 
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encoder is utilised to encode the segment according to its target encoding quality. 

The target segment bit rate of an initial segment is obtained from a user defined, target 
overall bit rate. After encoding the segment, the difference between the actual bit rate 
5 use d and the target segment bit rate is obtained. This difference is propagated to the next 
or next few segments to be coded. This process is repeated for each segment; therefore, 
for each subsequent segment, a new target segment bit rate is determined from the user 
defined target overall bit rate and the differences between the target segment bit rates and 
actual bit rates of previous or previous few segments. 

10 The present invention further provides a method for controlling bit allocation in a moving 
pictures encoder for encoding a sequence of images comprising a plurality of segments 
each having a plurality of images, the method including, for each segment: 

determining a difference between a number of bits used for encoding a previous 
15 segment and a number of bits allocated for encoding the previous segment; 

calculating a bits distribution from the determined bits difference and a 
predetermined distribution function; 

calculating a bit rate change from the bits distribution and a predetermined number 

of images in the segment; 
20 calculating a target segment bit rate from the bit rate change and a predetermined 

target overall bit rate for the sequence of images; and 

determining a target segment encoding quality from the target segment bit rate. 

The present invention further provides an encoding quality adjustment processor for 
25 generating a target segment encoding quality value in a moving pictures encoder for 
encoding a series of segments each having at least one image using a variable bit rate 
encoding scheme whilst maintaining a substantially constant overall bit rate, comprising: 

a bits difference computation means coupled to receive a segment encoding bit 
utilisation value and a target segment bit rate and generate therefrom a bits difference 
30 value representing a difference in bits allocated and bits used for encoding a segment; 
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a bits distribution means coupled to the bits difference computation means for 
computing at least one bits distribution value from the bits difference value and a 
predetermined distribution function; 

a bit rate difference computation means coupled to the bits distribution means for 
computing a segment bit rate difference from the at least one bits distribution value and a 
predetermined number of images in a segment; 

a target segment bit rate adjustment means coupled to the bit rate difference 
computation means and the bits difference computation means for computing said target 
segment bit rate from the segment bit rate difference and a predetermined target overall bit 
rate for the sequence of segments; and 

an encoding quality computation means coupled to the target segment bit rate 
adjustment means for computing a target segment encoding quality value from said target 
segment bit rate. 

15 The present invention further provides a moving pictures encoder comprising: 

a coding processor for encoding picture data based, on macroblocks according to a 
quantisation step size; 

a virtual buffer processor coupled to the coding processor for tracking a number of 
bits used for encoding successive macroblocks in a picture and a number of bits used for 
20 encoding successive pictures in a group of pictures; 

a quantisation step size processor coupled to the coding processor for determining 
said quantisation step size from a target number bits allocated for a picture and the number 
of bits already used for encoding macroblocks in that picture; 

a picture bit allocation processor coupled to the quantisation step size processor for 
25 determining said target number of bits allocated for a picture from a target bit rate and the 
number of bits already used for encoding pictures in a current group of pictures; 

a bit rate adjustment processor coupled to the picture bit allocation processor, the 
virtual buffer processor and the quantisation step size processor for determining said target 
bit rate from the number of bits already used for encoding successive pictures in the 
30 current group of pictures, a target encoding quantisation step size and an average 
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quantisation step size for pictures in the current group of pictures; and 

a target encoding quantisation step size processor coupled to the bit rate adjustment 
processor and the virtual buffer processor for determining said target encoding 
quantisation step size from a predetermined target overall bit rate and the number of bits 
5 used for encoding a preceding group of pictures. 

Brief Description of the Drawings 

The invention is described in greater detail hereinafter, by way of example only, with 
10 reference to the accompanying drawings, wherein: 

Figure 1 is a block diagram of the constant bit-rate controller based on TM5; 
Figure 2 illustrates the difference berween a target bit allocation and an actual bits 
consumption at macroblock level for one frame (a P-picture is chosen for this case), where 
is the virtual buffer fullness at a particular instance; 
! 5 Figure 3 is a block diagram of the variable bit-rate encoder with constant overall 

bit-rate control; 

Figure 4 is a block diagram of the constant overall bit-rate controller; 
Figure 5 illustrates how the difference in bit count for the last segment is 
redistributed over the next four segments (e.g. f(m) = l/4); and 
20 Figure 6 is a flow chart of a VBR algorithm with constant overall bit-rate control. 

Detailed Description of the Preferred Embodiments 

In a standard MPEG compliant video encoder, a sequence of moving pictures (e.g. video) 
25 is input to the encoder where it is compressed with a user defined target bitrate. The 
target bitrate is set according to the communication channel bandwidth in which the 
compressed video is to be transmitted, or the storage media capacity in which the 
compressed video sequence is to be stored. A typical MPEG encoder involves motion 
estimation/prediction, Inter/Intra classification, discrete cosine transform (DCT) 
30 computation, quantisation, zig-zag scanning, variable length coding and rate-control. 
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Several different forms of coding can be employed depending upon the character of the 
input pictures, referred to as I-pictures, P-pictures, or B-pictures. The I-pictures are intra- 
coded pictures used mainly for random access or scene update. The P-pictures use 
forward motion predictive coding with reference to previously coded I- or P- pictures 
5 (anchor pictures), and the B-pictures use both forward and backward motion 
predictive/interpolative coding with reference to previously coded I-or P-pictures. 
Furthermore, a group of pictures (GOP) is formed in encoded order starting with an I- 
picture and ending with the picture before the next I-picture in the sequence. 

10 The pictures are partitioned into smaller and non-overlapping blocks of pixel data called 
macroblocks (MBs) before encoding. Each MB from a P- or B-picture is subjected to a 
motion estimation process in which forward motion vectors, and backward motion vectors 
in the case of a B-picture MB, are determined using reference pictures from a frame 
buffer. The target macroblock in the current picture is matched with a set of displaced 

15 macroblocks in the reference picture, the macroblock that best matches the target 
macroblock is used as the predicted macroblock. The position of this predicted 
macroblock is specified by a set of vectors known as motion vectors, which describe the 
vertical and horizontal displacement between the target and predicted macroblock. For a 
B-picture, the process is similar except there are two reference pictures: anchor pictures 

20 immediately preceding and following the B-picture. 

With the determined motion vectors, motion compensation is performed where the intra-or 
inter-picture prediction mode of the MB is first determined according to the accuracy of 
the motion vectors found, followed by generating the necessary predicted MB. I-pictures 
25 are always intra coded while for P and B-pictures, a decision on whether intra or inter 
coding will be used (at macroblock level) is made depending on which method will give 
rise to a more efficient coding. 

Transformation of the macroblock using a DCT is then carried out on the 8x8 pixel blocks 
30 within the macroblock. For intra coding, the actual picture data is coded, while for inter 
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coding the prediction error is coded. This is followed by a quantisation process of the 
DCT coefficients which involves a quantisation matrix and a quantisation step size. The 
quantized coefficients are then run-length encoded with variable length codes. 

5 The resultant bit usage and statistical data are passed onto a rate control module, which is 
used for allocating a target number of bits used to encode each picture and each 
macroblock within the picture. This is an important module in the encoder as it plays a 
major role in maintaining the quality of the encoded picture. 

10 The rate control in MPEG-2 Test Model 5 (TM5) comprises the steps of allocating the 
target amount of bits for coding each picture, deriving the reference quantisation 
parameter (Q y ) to be used on each macroblock in a picture, and modulating the Q, based on 
the activity masking level of the surrounding blocks of the corresponding macroblock to 
obtain the modulated quantisation step size {MquaM) used to quantise the macroblock. 

15 

An objective of this rate controller is to ensure all pictures rnaintain a similar level of 
quality. It assumes that the subjective quality of a single coded picture can be qualified 
with a single number V, described by factor KJQ where K is a constant particular to a 
picture type and Q is the quantisation step size for the picture being coded. That is, the 
20 rate controller will try to preserve the following equation: 

O, Q p Q b 

The subscripts i, p, b refers to I, P, and B-picture types. The parameters K,, K p , and K b 
are experimentally determined constants for all I, P. and B pictures respectively, and K< 
usually is normalised to the value of 1 . 

25 Figure I illustrates a block diagram of a known embodiment of the TM5 rate controller. 
A group-of-pictures (GOP) is a collection of one I picture, some P pictures and B pictures, 
and serves as a basic access unit with the I picture as the entry point to facilitate random 
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access. The controller comprises three levels of processing: a GOP level 111. a picture 
level 1 10, and a macroblock (MB) level 109. At the start of every new GOP at the GOP 
level 111. a GOP bit allocation process 100 computes the total number of bits (R (op ) 
allocated for the GOP. The value of R top is given by: 

bit rate x N 



sop 



picture _rate 



(1) 



5 where bitjate is the target bit rate for encoding the picture sequence, 

picture _rate is the number of pictures coded per second, and 
N is the total number of pictures coded in the GOP. 

The remaining bits {R) for the GOP is determined at the picture level 110 by a remaining 
10 bits determination process 106. The value of R is updated as: 



R=R-S 



R = R + R 



where S is the number of bits used by previously coded 
pictures in the GOP, and 
for a new GOP. 



15 



With the computed value of R, a picture bit allocation process 101 at the picture level 110 
then computes a target bit value (7) allocated to the current picture according to the 
equations below for corresponding I, P or B picture type: 

R bit_rate 



T, = max 



^ NJC p N b X b ' K x picture jrate 



(2) 



T. = max 



bit rate 



N k K X h ' K x picture jrate 



(3) 
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T b = max 



bit rate 



NKJC n ' K x picture _rate 

N, ■ p " 



(4) 



where 

T„ T p . T b Represents target bits for the next picture (I.P.B) 

N bt N p Represents the remaining number of pictures (P,B) in the GOP 

X t , X ? , X b Represents global complexity measures and gives a measure of the 

actual number of bits required to represent the picture (without 

compression), and 



X f = Q, x S; 

10 *,-Q,*s, 

X b = O, x S b 



S„ S„, S b are the actual number of bits used to code the previous I/P/B 
frame. 

15 k is a constant (e.g. 8) 



After compuung the target bits for the picture, a reference quantisation step-size (Gy) 
computation 102 is carried out at the MB level 109 for each macroblock within the 
picture. The value of Q, is computed based on the determined target bit allocation (7% T p , 
20 or T b ) and a buffer fullness value (dp. Each of the three picture types has a virtual buffer 
associated with it and these buffers are updated after coding each macro block of the 
picture type according to the following equations: 
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(6) 



d 

j 



■ 4? + By-i" 



r t x CM) 

MB Of 



(7) 



where rf„ is the initial virtual buffer fullness, 

dj is the virtual buffer fullness when coding the/ MB, 

is the actual bits consumed up to and including </-D* MB as provided 
5 by a virtual buffer update process 105, and 

MB_Ctu is the number of MBs in the picture. 

Equations (5) to (7) effectively track the differences between the actual number of bits 
used and the target bits allocated, and these differences are then added to the respective 
10 virtual buffers which are used to compute Q, This allows the rate controller to control the 
bits allocation based on the consumption pattern of the picture. Q, is computed from the 
buffer fullness via the equation: 

<2 ; = {dj x 31) / r = dj x constant 
15 where r is a reaction parameter = 2 x bitjaie I picture jrate 

The reaction parameter, as the name implies, is a factor that can control the sensitivity of 
the algorithm from a change in d. Large r will cause the reaction to be slower, which 
may cause the target bits and actual bits to differ significantly, but it brings about a more 
20 gradual rate of change in bit consumption which is favourable for scene changes. On the 
other hand, a small r causes the controller to be more reactive to changes, giving rise to a 
closer target and actual bits value, but a faster response also means a less gradual change 
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during scene change which is not desirable. A graph illustrating the difference between 
the target bit allocation and the actual bits allocated within a frame is shown in Figure 2. 

The value of Q, is further modified by an activity masking process 103 to give the 
5 modulated quantisation step size, Mquant. Basically, the spatial masking ability ot the 
macroblock is used to place a correction factor onto the computed Qj value. Spatial 
masking ability is the ability to contain noise masked from human visual systems. 
Typically, an area with complex texture will have a larger correction factor than an area 
with simple texture, hence, 



10 



< 9 > 

Mquant = Njict, x Q, 

where N_act } is the normalised value of macroblock activity level and may have a 
value of between 0.5 and 2 



15 



With the computed Mquant, the macroblock coding process 104 is performed where the 
macro block is compressed according to the MPEG standards. 

,0 At the end of coding a picture, the bit count S used for coding the picture is used to 

perform a VBV check 107 and to compute the rernaining bits R for the current GOP at the 
remaining bits determination process 106. The VBV (video buffer verifier) is a virtual 
buffer emulating the status of the decoding buffer. The VBV check (107) is performed to 
detect any overflow or underflow of the decoding buffer with reference to the target bit 
25 rate. 

When coding a picture sequence with varying scene complexity-, a constant bit rate (CBR) 
encoder is unable to vary the bit-rate according to the scene complexity. Tins results in 
scenarios whereby simple scenes are allocated more bits than required while complex 
30 scenes are allocated insufficient bits, and therefore results in a variation of output visual 
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quality between scenes of different complexities. 

An embodiment of a moving pictures encoding apparatus according to the present 
invention is illustrated in block diagram form in Figure 3. The encoding apparatus as 
5 illustrated is arranged in four levels corresponding to processing stages in the bit rate 
control process, namely a segment level 313, a GOP level 312, a picture level 311 and a 
macroblock (MB) level 310. An input moving pictures sequence is encoded in such a way 
that on the average, or at the end of encoding, the output bit rate is close to a user 
definable target overall bit rate (overall_BR) 320. The input sequence is divided into 
10 segments and groups of pictures (GOP). The purpose of a segment is for monitoring of 
output bit rate with reference to the overall_BR, and the purpose of the GOP is primarily 
to facilitate random access. A segment may be defined to include a number of GOP(s) 
depending on the need in terms of frequency of monitoring output bit rate. Each GOP 
comprises at least an I-picture and optionally one or more P-pictures and/or B-pictures. 
15 Each picture is divided into macroblocks of pixels for encoding. Hence, a method of rate 
control according to the current invention may contain processes at the segment level 313, 
GOP level 312, picture level 311, and finally macroblock level 310. 

With a new group of pictures, a GOP bit allocation processor 300 computes the number of 
20 bits allocated to the current GOP (R' top ) as follows: 



R =BRx £^£E (10) 

gop picture jrate 



where: 

N gop is the number of pictures in the GOP, 
picturejaxe is the number of pictures coded per second, and 
25 BR is a target bit rate. 



The target bit rate BR is determined for each picture or plurality of pictures by a bit rate 
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—eat processor 306 which is described hereinbelow. A remaining number of bits 
<*■> for the GOP is determined at the picture level 311 by a remaining bits determinant 
processor 308, which carries out the steps of: 



5 (a) 



before encoding the first picture in a GOP, adjusting *' with a new R W 
R' + = R' lop , and then set BR old = BR 



<b> otherwise if the picture to be coded is not the first picture of a GOP, and a new 
target bit rate BR is determined, then adjusting R' with the new BR according to: 

R „ E x (BR - BR M ), and then set BR M = BR 

picture jrate 

where N is remaining number of picmres in rhe current GOP 

(c) and removing number of hi* used by the .asr coded picture S from the remaining 
bits value R', hence: 



15 



R' =R'-S 



The computed R' is then passed to a picmre bit allocation processor 301 to compu* the 
target bit O) allocated to a current picmre to be coded. For example, eolations (2), (3) 
20 and (4) described hereinabove may be used for that purpose, with the parameters K b , 
Np , being supplied. Note that the lower limit of «/(8 W .rot,) 

is only optional and may be adjusted if necessary. 

AI the macroblock (MB) level 310. a reference quantisation step-size conjuration 
n processor 302 computes a reference quantisation srep-size Q, for each MB ustng the 
" computed target bit location T and a bit utilisation partner 8 determ.ned by a vtnual 

buffer update processor 305. An example of a merhod for computing Q, for the J MB ot 
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the current picture is represented by: 



AT, x Dj x picture _rate 
-J = K 2 x BR 

where K x and K 2 are constants (eg. 31 and 2 respectively), 

BR is the determined target bit rate, 

picture_rate is the number of coded pictures per second, and 
Dj is a virtual buffer fullness 

The virtual buffer fullness of the / MB is determined by: 

D = D n * B — - ( l2 > 

where D 0 is an initial buffer fullness before coding the current picture, ie. Dj at end of the 
previous picture, 

B is the bit utilisation information supplied by the virtual buffer update processor 

305, ie. bits used to code the 1 st MB to (j-kji* MB, 

MB_Cnt is the total number of MB in the current picture, and 

K 3 is a constant (eg. 1). 

5 Three independent current and initial virtual buffers fullness values (£>/, Df, £>/, A,', DJ, 
Do") may be maintained for the three picture types (I-pictures, P-pictures, and B-pictures). 
The computed Q, may also be further scaled by an activity masking processor 303 
according to a masking factor determined by the surrounding activity levels of the MB to 
form the final quantisation step-size, Mquant, for coding the current MB. A method of 

0 masking factor determination can be found in the aforementioned MPEG-2 TM5. 

Macroblock coding (304) is then performed to encode the current MB. A macroblock 
coding processor 304 may employ, for example, methods according to the MPEG-1 or 
MPEG-2 video encoding standards. Such encoding methods include necessary motion 
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compensation, discrete costne transform, quantisation with the determined quanusatton 
step size, and nm-length encoding wtu variable length codes. The methods a.*, tnclude 
necessary decoding processes such dm motion compensation can be performed^ The 
number of bits utilised by me macrobiock coding processor 304 ,0 code each MB ,s passed 
5 to the virtual buffer update processor 305. 

to applications where a maximum and/or minimum bit rare musr be mainlined for 
encoding, a VBV checking processor 307 is utiUsed. User definable maximum and/or 
tuinimum bit « <t— BR) are input a. 32! to me VBV checker 307. a method of 
10 VBV checking accordtng to the MPEG standards is used » examine t* utilisauon 
information from the virtual buffer 305, and necessary corrections are made to ensure 
compliance and that tire output bit rate is within the defined max/min BR. 

At the end of coding a picture or a plurality of picmres. bit rate adjustment U applied by 
,5 the bit rate adjusunent processor 306 to provide any necessary correction . *e current 
target bit rate BR. The correction is based on a target encoding quality provtded by a 
target segment quality adjustment processor 309 and a resultant encoding qoahty of 
previously coded pictured such that tire overall encoding quaUties of picmres wrthtn a 
segment are reiativdy Cose to the targe, consent. The resultant encoding quality may he 
20 determined by the average value of the reference quantisation step size (ie. am*._&-<* 
*. previously coded pictured A method of target bit rate {BR) determination can be 
derived from a rate-quantisation model as given by: 

K A x (meragej}, - targetj)) 
BR =■ current BR * average^, 

where current_BR is a current estimated bit rate, or 
current _BR = S, + N, x S, + N, x S„ 
25 S„ S,. I are bits used by previously coded I. P. and B-pictures respecuvely. 

N r . N. are total number of P and B-pictures in the current GOP, 
averagej}, is the average value of Q, of previously coded picrure<s) 
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target Q, is the target encoding quality or target value of Q,, 

K A may be a constant, or a factor of previous BR, max JR. or current JR, and K 4 

may also be separately determined for I, P. and B-picture types. 

5 Maximum and/or minimum bit rates can be applied to the determined BR according to 
application requirements. The max/min BR input at 322 are used as given by: 

if (BR > maxJR), then BR - maxJR 
if (BR < minJR), then BR = minJR. 

10 

It is also possible to make use of the target J2, at the macro-block level to increase 
efficiency of encoding. Basically, the target_Q j is used by the reference quantisation step- 
size computation processor 302 as a lower limit of the output reference quantisation step- 
size Qj such that when this target (quality) is reached at the MB level, bits are saved for 
15 future encoding immediately. In this case, the output reference quantisation step size may 
be set according to: 

if (Qj < targetjQ,), then Q } = target _Q y . 

20 The target bit rate BR is adjusted continuously with reference to a fixed target encoding 
quality within each segment. In turn at the beginning of each segment, the target 
encoding quality is adjusted so that a target overall bit rate is achieved. A segment may 
contain a few groups of pictures depending on the need in terms of frequency of 
monitoring output bit rate. A new segment may also be defined by a scene change 
25 detectable by any conventional scene change detector with a given range of time. 

At the onset of a new segment, the target segment quality adjustment processor 309 
executes a check on the overall bit-rate based on bits usage and makes any necessary 
adjustments to the target encoding quality for the bit rate adjustment processor 306 so that 
30 the overall bit-rate converges to a user definable target overall bit-rate 320 (overall JR). 



SUBSTITUTE SHEET ( rule 26 ) 

JSDOCID: <WO 9949664A1J_> 



WO 99/49664 



PCT/SG98/00022 



- 19- 

An example embodiment of a constant overall bit rate controller implementing the target 
segment quality adjustment process according to the present invention is illustrated » 
block diagram form in Figure 4. The bit count difference between the actual and target btt 

(bits Jiff) for coding of a previous segment is calculated by a bits difference 
computation processor 401, utilising the steps of: 



counts 

5 



a) 



obtaining the actual bits (bitsjegment) used for coding the previous segment 
according to the input bit utilisation of pictures (S) 400 in the previous segment. 



and 



10 

b) 



computing the value of bits Jiff based on a corresponding target segment bit rate 
406 (segment_BR) for the previous segment, according to: 

t . ..~ segment _BR x N 3egmem _ segment (14) 
= picturejate 

where N^is the number of coded pictures in the previous segment. 



15 The bit count difference (bits Jiff) is then redistributed by a bit difference distribution 
processor 402 over the next k number of segments using a bit distribution function,*/*), 
expressed as: 

delta_bits„ = f(m) x bits Jiff 

20 

where m — \,...,k, 

delta bits m is the bits difference distributed to next nf segment. 

25 For an example case of *=4 and Am) = Ilk, Figure 5 illustrates how the bit count 

difference (bits diff) of an encoded segment is distributed over the next 4 segments. The 
period of k segments used for bits compensation is referred to as an adjustment interval. 
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A delta bit-rate computation processor 403 (Figure 4) accumulates all bit differences 
(delta Jits) distributed from previously encoded segments to the current segment to be 
coded, and computes a delta segment bit-rate (AsegmentJR) based on the accumulated 
delta Jits and the number of pictures in the current segment to be coded. The delta 
5 segment bit-rate and a user definable target overall bit-rate (overallJR) input at 408 are 
used by the target segment BR adjustment processor 405 to derive the target segment bit 
rate (segment JR) 406 for the current segment. The value of segment JR can be 
expressed as: 

10 segment JR = overall_BR + ^segment JR < 16 ^ 

A new target encoding quality (target J® 407 for encoding of the current segment is then 
determined from the target segment bit rate using a BR-Qj modelling processor 404. The 
BR-Qj modelling processor may operate, for example, according to: 

(segm ent BR - segmentJR') 

AO. = target Q, x — pp , (1 ') 

-J 6 — J K.x segment _BR 



target _O f = target _O y + AQ y (18) 



15 where target jQ) is the target Q, of the previous segment, 

segment JR' is the target segment bit rate of the previous segment, and 
K 5 is a constant which can be experimentally determined. 

A maximum limit may be placed on the determined A<2 y so that there is no drastic change 
20 in output quality from one segment to another. 

A flow chart for a process of variable bit rate (VBR) encoding with constant overall bit- 
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rate control according to an embodiment of the invention is illustrated in Figure 6. 
initialisation of predetermined parameters is first carried out at step 600. Then, at step 
601 the process determines whether processing is at the start of a new group-of-pictures 
(GOP) or not. Where a new GOP is determined, step 602 is carried out by updating the 
5 bit allocation for the new GOP. This may involve, for example, computation of the GOP 
bit allocation R'„ according to Equation (10) described hereinabove and accumulating to 
the remaining bit value R' to give the R' for the new GOP. 

For each picture in the current GOP, a target bit allocation value T is determined at step 
10 603, and this may be computed according to Equations (2), (3) and (4), for example. 
Then for each macroblock in the picture, a reference quantisation step size Q is 
determined using the computed target bit allocation T, such as by the process represented 
by Equations (11) and (12). Activity masking may also be included in this step. The 
macroblock of the picture is then encoded at step 605 using the computed quantisation step 
15 size, and steps 604 and 605 are repeated for all of the macroblocks in the picture until the 
end of the picture is determined at step 606. 

When the end of the sequence of moving pictures which is being encoded is reached, this 
is determined at step 607, which terminates the process at step 611 upon that occurrence. 
20 Otherwise, the process continues to step 608 where it is determined whether the end of the 
current segment of pictures has been reached. 

When the end of a segment of pictures is reached and processing of a new segment about 
to begin, the target encoding quality is adjusted at step 609. This is performed by firstly 
25 computing the difference between the number of bits allocated for coding the previous 
segment and the actual number of bits used in coding that segment, such as by the process 
represented by Equation (14). This quantity, bits Jiff, represents extra bits which are left 
over from the previous segment of pictures, and may be distributed for use in encoding the 
ensuing picture segments. The left over bits bits Jiff ^ distributed for use over one or 
30 more segments according to a bit distribution function, an example of which is described 
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in connection with Equation (15). In essence, the left over bits from the previous segment 
are divided into a plurality of fc preferably equal, amounts and allocated to the following * 
segments for encoding with. The dividends allocated from all of the k previously 
processed segments are accumulated and used to determine a change in segment to rate, 
5 ±se g meM_BR. according to the accumulated bits difference and the number of pictures in 
the segment to be coded. This can then be used to compute an allocated bit rate for the 
segment to be coded according to Equation (16). Finally, a target encoding quality, 
target_Q f , can be computed according to Equations (17) and (18), for example. 

10 If the end of a segment has not been reached at step 608, or after the target quality 
adjustment of step 609, the target quality and the actual quality of encoded pictures is 
compared, and a new bit rate for encoding the next picture is computed based on the 
difference. This step can be carried out as described in relation to Equation (13), for 
example. The procedure then returns to step 601, described above. 

^ In summary, embodiments of the present invention provide methods and apparatus for 
encoding moving panares with a variable bit rate whilst maintaining a consistent output 
encoding quality according to a determined target quality. The target quality is adjusted at 
the beginning of every moving pictures segment according to a bit distribution function to 
20 ensure an overall bit rate that meets a defined target. Furthermore, the segment based bit 
distribution function provides smooth and flexible modulation of the encoding quality. 
The method can be implemented at low cost and with the ability to encode an input 
moving pictures sequence in one-pass at real-time. 

25 u will be readilv apparent to those of ordinary skill in the art that the foregoing detailed 
description of the present invention has been presented by way of example only, and is not 
intended to be considered limiting to the invention as defined in the claims appended 
hereto. In particular, it is envisaged that numerous variations to the embodiments as 
described can be made without departing from the spirit and scope of the invention. 



0 
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Throughout this specification and the claims which follow, unless the context requires 
otherwise, the word "comprise", and variations such as "comprises" and "comprising" . 
will be understood to imply the inclusion of a stated integer or step or group of integers or 
steps but not the exclusion of any other integer or step or group of integers or steps. 



5 
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Glossary: 



VBV video buffer verifier 

VBR variable bit rate 

5 CBR constant bitrate 

MPEG Moving Picture Experts Group 

DCT discrete cosine transform 

GOP group of pictures 

MB macroblock 

i0 Q quantisation parameter (picture quality) 

q quantisation parameter for macroblock j 

Q h Q b * Q P quantisation parameters for I, B and P type pictures 

r number of bits allocated for encoding a group of pictures 

bitjaxe target bit rate for a picture sequence 

1 5 picture jote number of pictures coded per second 

N total number of pictures in the group of pictures 

Ak fif remaining number of P,B pictures in the group of pictures 

r 9 r ' remaining number of bits for encoding a group of pictures 

S number of bits already used in coding a group of pictures 

20 r„ T r T b target number of bits for coding next I, P, B type picture 

X if Xp, X b global complexity measures for I, P, B type picture 

d D virtual buffer fullness when coding the/ 1 macroblock 

d 0 , D Q initial virtual buffer fullness 

B actual bits used up to and including (H)* macroblock 

25 MBjCnt number of macroblocks in the picture 

r reaction parameter ( = 2 x bitjrate I picture jrate) 

N act; normalised macroblock activity level 

Mquant modulated quantisation step size (= Njictj x Qj) 

overall JBR definable target overall bit rate 

30 BR target bit rate 
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B bit utilisation information (eg. bits used to code previous MBs in picture) 

current BR current estimated bit raw 

averag'ejl, average value of quantisation parameter for previously coded pictures 

target J2j ' quantisation parameter value for target encoding quality 

5 max BR maximum permitted bit rate 

min BR minimum permitted bit rate 

bits Jiff difference between actual and target bits used in coding previous segment 

segment _BR target segment bit rate 

bitsjegment actual number of bits used in coding previous segment 



10 N. 



number of coded pictures in previous segment 



delta _bits m number of difference bits available to next m* segment 
j{ m ) bits distribution function 

Asegment_BR available difference in segment bit rate 
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Claims: 

1. A method for use in a moving pictures encoder for encoding a sequence of 
segments each having at least one image, comprising the steps of: 
5 a) determining an overall target bit rate for encoding the sequence of images; 

b) determining a bit allocation and target quantisation step size for encoding a 
first segment on the basis of a segment target bit rate calculated using said overall 
target bit rate; 

c) encoding said first segment using a variable bit rate encoding method 
10 according to the target quantisation step size; 

d) determining a difference between the number of bits used to encode said 
first segment and said first segment bit allocation; 

e) distributing said difference for use in encoding at least one subsequent 
segment to determine a subsequent segment bit allocation; 

1 5 f) determining a new target quantisation step size for encoding a said 

subsequent segment on the basis of a new target segment bit rate calculated using 
said segment target bit rate and the distributed difference; and 
g) encoding said subsequent segment using a variable bit rate encoding method 
according to the new target quantisation step size; 

20 wherein variable bit rate encoding is employed for encoding pictures within a 

segment whilst maintaining a substantially constant bit rate over said sequence. 

2. A method as claimed in claim 1 , wherein the steps d) to g) are carried out 
iteratively using the number of bits used to encode the subsequent segment and the 

25 subsequent segment bit allocation for determining said difference. 

3. A method as claimed in claim I or 2, wherein each segment comprises at least one 
group of pictures having an l-pieture and optionally at least one P and/or B-picture. 

30 4. A method as claimed in claim 2 or 3, wherein the difference between the number 
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of bits used to encode the first segment and said first segment bit allocation is determined 
according to: 

__ overall_BR x N„ %menl _ bitsjegment 
- picture jraie 



where: 



bitsjiiff is said difference, 

5 overall_BR is the overall target bit rate, 

m is the number of coded images in the first segment, and 

Ly segment 

bitsjegment is the actual number of bits used to encode the first segment. 



10 to : 



A method as 



claimed in claim 4, wherein said difference is distributed according 



delta_bits m = fim) x bits Jiff 
where delta JUs a is the bits difference distributed to next m* segment in the sequence, 
m — 1 , • • - >^-» 

k is a positive integer, and 
15 fim) is a bit distribution function, where Tffrm) = 1. 

6. A method as claimed in claim 5, where /(m) = Vk. 

7 A method as claimed in claim 5 or 6, wherein the steps d) to g) are carried out 
20 iterative* using the number of bits used to encode the subsequent segment and the 
subsequent segment bit allocation for determining said difference. 

8 . A method as claimed in claim 7. wherein the new target quantisation step size is 
calculated according to: 

(segment BR - segment _BR') 
AO = target_Q x = segment _ BR 

25 
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target J) = target J? + AO 

where target JQ is the new target quantisation step size, 
segment JBR is the new target bit rate, 

target JQ' is the target quantisation step size of the preceding segment, 
segment _BR' is the target segment bit rate of the preceding segment, and 
5 AT is a constant. 

9. A method as claimed in claim 7 or 8, wherein for each subsequent segment the 
difference between the number of bits used to encode the segment and the segment bit 
allocation is determined according to: 

segment BR x N Mmmmmtt 

bits diff = — = - bttsjsegment 

~ picture j-ate 

10 where: 

bitsjliff is said difference, 

segment J2R is the segment target bit rate, 

N _ is the number of coded images in the segment, and 

1 segment 

bitsjsegment is the actual number of bits used to encode the segment. 

15 

10. A method as claimed in claim 9, wherein said difference is distributed according 
to: 

delta J>its m = j{m) x bitsjliff 
where delta J>its m is the bits difference distributed to next m? segment in the sequence, 
20 m = 

k is a positive integer, and 

J{m) is a bit distribution function, where S k ^m) = 1. 
11. A method as claimed in claim 10, where J{m) = l/k. 

25 
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U A method for encoding moving pictures in a moving pictures encoder wherein a 
sequence of images are provided as input, the sequence of images comprising a plurality 
of segments each having a plurality of images, the method including: 

a) determining an overall target bit rate for encoding the sequence of images; 
5 b ) maintaining a distribution record of bits from at least one previously 

encoded segment allocated for use in encoding at least one segment to be encoded; 
c) detemiining a target segment bit rate for a segment of the sequence of 
images on the basis of the overall target bit rate and a bit rate change calculated 
from the corresponding allocated bits from the distribution record; 
l0 d) detemuning a target segment encoding quality from the target segment bit 

rate, a preceding target segment bit rate and a preceding target segment encoding 
quality,; and 

e) encoding the images of the segment according to the target segment 
encoding quality using a variable bit rate encoding technique taking into account 
1 5 scene complexities of the images in the segment; 

wherein maintaining said distribution record includes determining a difference 
between the number of bits used to encode a particular segment and the number of bits 
allocated for encoding the particular segment on the basis of the target segment encoding 
quality. 

20 . 

13. A method as claimed in claim 12, wherein steps b) to e) are carried out iteratrvely 

for said sequence of segments. 

14. A method as claimed in claim 13, wherein each segment comprises at least one 
25 group of pictures having an I-picture and optionally at least one P and/or B-picture. 

15. A method as claimed in claim 12, 13 or 14, wherein the target segment encoding 
quality is calculated according to: 

~ (segment BR - segment _BR) 
AO = target O x E= _ - — 
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target_Q = target_0' + AO 

where target jQ is the target segment encoding quality, 
segment _BR is the target segment bit rate, 
target _Q' is the preceding target segment encoding quality, 
segment _BR' is the preceding target segment bit rate, and 
5 AT is a constant. 

16. A method as claimed in claim 15 , wherein the difference between the number of 
bits used to encode a particular segment and the number of bits allocated for encoding the 
particular segment is determined according to: 

bits diff m segment_BR x N segmenl _ Utsjegmmt 
- picturej'ate 

10 where: 

bits JLiff is said difference, 

segment JBR is the segment target bit rate t 

at is the number of coded images in the segment, and 

bitsjegment is the actual number of bits used to encode the segment. 

15 

17. A method as claimed in claim 16, wherein the allocation of bits in the distribution 

record is calculated according to: 

deUa_bits m = f{m) x bits Jiff 
where delta bits m is the bits difference allocated to next m* segment in the sequence, 
20 m = 1, 

k is a positive integer, and 

fim) is a bit distribution function, where S k fi,m) = 1. 
18. A method as claimed in claim 17, where J{m) = \/k. 

25 
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19 A method for controlling bit allocation in a moving pictures encoder for encoding a 
sequence of images comprising a plurality of segments each having a plurality of images, 
the method including, for each segment: 

determining a difference between a number of bits used for encoding a previous 
5 segment and a number of bits allocated for encoding the previous segment; 

calculating a bits distribution from the determined bits difference and a 
predetermined distribution function; 

calculating a bit rate change from the bits distribution and a predetermined number 

of images in the segment; 
10 calculating a target segment bit rate from the bit rate change and a predetermined 

target overall bit rate for the sequence of images; and 

determining a target segment encoding quality from the target segment bit rate. 

20 A method as claimed in claim 19, wherein the difference between the number of 
13 bits used for encoding the previous segment and the number of bits allocated for encoding 
the previous segment is determined according to: 

bits d iff = Segme " l - BR X N ' egmem ~ bits_segment 
- picture jrate 



where: 

bits_diff is said difference, 

segment_BR is the previous segment target bit rate, 

, n N is the number of coded images in the previous segment, and 

ZU segment 

mjegmem is the actual number of bits used to encode the previous segment. 

21. A method as claimed in claim 20, wherein the bits distribution is calculated 
according to: 

25 delta bits m = J{m) x bits Jiff 

where delta bits m is the number of bits difference allocated to next m* segment in the 
sequence, 
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m — 1> — »^» 

k is a positive integer, and 

J{m) is a bit distribution function, where 2 k .ftm) = 1. 

22. A method as claimed in claim 21 , where fljn) = Vk. 

23. A method as claimed in any one of claims 19 to 22, wherein the target segment 
encoding quality is calculated according to: 

(segment BR - segment _BR') 
AO = target O x - 5ggment _ BR 

target _0 = targetj? * 



where targetjQ is the target segment encoding quality, 
10 segment _BR is the target segment bit rate, 

targetjQ' is the previous segment target segment encoding quality, 
segment _BR' is the previous segment target segment bit rate, and 
K is a constant. 

1 5 24. An encoding quality adjustment processor for generating a target segment encoding 
quality value in a moving pictures encoder for encoding a series of segments each having 
at least one image using a variable bit rate encoding scheme whilst maintaining a 
substantially constant overall bit rate, comprising: 

a bits difference computation means coupled to receive a segment encoding bit 
.0 utilisadon value and a target segment bit rate and generate therefrom a bits difference 
value representing a difference in bits allocated and bits used for encoding a segment; 

a bits distribution means coupled to the bits difference computation means tor 
computing at least one bits distribution value from the bits difference value and a 
predetermined distribution function; 
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a bit rate difference compuution means coupled . the biu distribution means for 
computing a segment bit tate difference from me a. ieas. one biu distribution vaiue and a 
prederermined number of images in a segment; 

a target segment bit rate adjustment means coupled to the bit rau dtfference 
, compuution means and the bits difference ccmputadon means for compudng said target 
segment bit rate .Tom dte segment bit rau difference and a predefined target overall bt, 

rate for the sequence of segments; and 

an encoding quality computation means coupled <o the urge, segment b.t rae 
aojusunen, means for compudng a urge, segmen. encoding quality value from said urge. 
10 segment bit rate. 

25 A moving pictures encoder comprising: 

a coding processor for encoding picmre dau based on macroblocks according to a 

Quantisation step size; 

.. a virma. buffer proce^rcoup.ed.o me coding processor for «cfcmg a number of 

bits used for encoding successive macrobloclts in a picmre and a number of biu use4 for 
encoding successive pictures in a group of pictures; 

a quantisation sup size processor coupled » to coding processor for deumnnmg 
said quantisation step size from a urge, number bus allocated for a picmre and me number 
20 of biu already used for encoding macroblocks in that picmre; 

a picmre bit allocation processor coupled to the quantisation sup size processor tor 
deutrmning said urge, number of biu aliocaud for a picmre from a urge, bit rate and the 
number of bits already used for encoding pictures in a current group of ptccures; 

a bit rate adjustment processor coupled . the picmre bi. aUocanon processor, dte 
,5 vtttual buffer processor and dte quantisation step size processor for deutmining s,d urge, 
bit rate from the number of biu already used for encoding successive picmres m dte 
curten. group of picmres. a urge, encoding quantisation s.ep size and an average 
quantisation step size for picmres in the current group of picmres; and 

a urget encoding quantisation step size processor coupled <o me b,t rate ad.us.men, 
30 processor and the virtua. buffer processor for deurmining said urget encodmg 
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quantisation step size from a predetermined target overall bit rate and the number of bits 
used for encoding a preceding group of pictures. 
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